The invention relates to image processing and, more particularly, to processing volumetric image data of a semitransparent object that generates a shadow.
Texture-based volume rendering is a technique to efficiently visualize volumetric data using texture mapping hardware.
In volume rendering, the volume data are typically treated as a semitransparent cloud of voxels (units of volumetric image data) emitting diffuse light. The task is to find per pixel the total intensity of all voxels that contribute to it. The most straightforward method casts a ray through each image pixel and integrates the intensities of all voxels pierced by the ray. Care must be taken to weigh the contribution of voxels further away by the accumulated opacity of the voxels in front.
Otherwise, all voxels of the same luminance have the same brightness on screen and it would be practically impossible to distinguish front from back. Unfortunately, the ray-casting approach is computationally expensive and usually does not allow interactive manipulation of the data. To overcome this limitation, alternative methods have been developed that achieve comparable results in less time, of which texture-based volume rendering promises the highest interactive frame rates.
The use of three-dimensional textures to perform volume rendering was described by Cabral et al., Accelerated volume rendering and tomographic reconstruction using texture mapping hardware, 1994 Symposium on Volume Visualization, pages 91-98. The method differs from so-called image-based volume rendering methods which calculate the final image pixel by pixel, by its object-based approach and its extensive use of image composition and texturing hardware. Whereas image-based methods integrate luminance and opacity over a ray shot through the volume of interest in software, texture-based methods render the data as a stack of textured parallel slices, from back to front, leaving the integration of luminance and opacity to an image composition step that can efficiently be performed in hardware. This speed advantage is impaired by the inferior quality of the created images. Important secondary visual cues like illumination and shadows were formerly hard to integrate at interactive frame rates, if at all.
This is unfortunate, especially in applications like medical visualization, where a thorough understanding of the spatial relationships is crucial. In the context of a neurosurgical operation planning system, we observed that the image quality of texture-based volume rendering often does not sufficiently communicate the orientation of the data-set. While this can partly be overcome as long as the object is moved, the impression vanishes as soon as a still image is examined in closer detail. We therefore searched for methods that add visual cues to the image without spoiling interactivity.
The basic method for rendering volume data with texture mapping works as follows. First, the three-dimensional data-set is loaded into the texture buffer of the graphics subsystem as a three-dimensional texture block. An image is generated by rendering a stack of parallel planes, perpendicular to the viewing direction, from back to front, with each plane being textured with an accordingly oriented slice from the texture block. Each new texture plane is then blended with the contents of the framebuffer by using a suitable blending function. The back-to-front rendering ensures a proper depth relation between the slices, while the purpose of the blending is to ensure that voxels farther away appear with an intensity scaled by the transparency of the voxels in front.
A number of extensions have been proposed to the basic method. Variations of the blending operation allow for different properties to be extracted from the data. Examples are the common xe2x80x9coverxe2x80x9d operator which implements attenuated integration, similar to image-based volume rendering, or a maximum intensity projection (MIP) operator which can be used to visualize contrast media in medical imaging.
Van Gelder and Kim, in Direct Volume Rendering with Shading via Three-Dimensional Textures, 1996 Volume Visualization Symposium, pages 23-30, extended the method to incorporate diffuse and specular illumination. In a preprocessing step, their method finds the gradients at each voxel. When rendering the volume, the gradient serves to calculate the total light intensity of each voxel. These results are then saved as a new 3D texture that can be rendered. Although this method gives impressive results in terms of additional realism, and improved spatial comprehension can be achieved, it decreases rendering performance on the order of a factor of 10.
These modifications are motivated by the need to improve the visualization with respect to selected aspects of the data. The MIP-variation of the composition operator is used to focus attention towards the area of strongest contrast enhancement, while van Gelder and Kim try to overcome the foggy, dull appearance of the original method and emphasize the underlying isoluminance surfaces and thus the spatial context.
An important secondary visual cue for orientation in a volume are shadows. The superior visual impression of volume data rendered by using ray tracing techniques is caused by the detailed modeling of illumination, including shadows. In the context of a neurosurgical operation planning system, we found that shadows can significantly improve the spatial understanding of volume data, if rendered at interactive frame rates.
The central problem to be solved here is how to decide whether a voxel is shadowed by another voxel. A voxel is in shadow with respect to a light source if any other non-transparent voxel lies on the line of sight between itself and the light source. Thus, to decide whether a voxel is in shadow, a ray is traced to the light source, and the opacity of all intersected voxels is integrated. The amount of light reaching the final voxel is inversely proportional to the accumulated opacity. This technique is known as xe2x80x9cshooting shadow raysxe2x80x9d in ray tracing. A similar, preprocessing version is at the core of Reeves""s shadow map method in Rendering Antialiased Shadows with Depth Maps, Computer Graphics (SIGGRAPH ""87 Proceedings), volume 21, pages 283-291, July 1987. In Volumetric Ray Tracing, 1994 Symposium on Volume Visualization, pages 11-18, Sobierajski presented an illumination model for volume rendering taking into account shadows, scattering and other subtle effects, which was successfully used in a state-of-the-art system. Earlier, Rushmeier, in The Zonal Method for Calculating Light Intensities in the Presence of a Participating Medium, Computer Graphics (SIGGRAPH ""87 Proceedings), volume 21, pages 293-302, July 1987, developed a similar model, although her approach was based on radiative heat transfer.
In xe2x80x9cMethod to Generate the Shadow of Arbitrary-Shaped Three-Dimensional Objects,xe2x80x9d IBM Technical Disclosure Bulletin, vol. 33, no. 2, Jul. 1, 1990, pages 146-148, a method for generating the shadow of an arbitrary-shaped three-dimensional object is disclosed wherein the object is totally opaque. Moreover, E. A. Segal, in xe2x80x9cFast Shadows and Lighting Effects Using Texture Mapping,xe2x80x9d Computer Graphics, vol. 26, no. 2, July 1992, pages 249-252, discloses a method for generating shadows and lighting effects using texture mapping wherein again the objects displayed are opaque.
The present invention relates to a method that extends this approach to render shadows for the volume. The method takes advantage of fast framebuffer operations modem graphics hardware offers, but does not depend on any special purpose hardware.
The visual impression of the final image is significantly improved by bringing more structure and three-dimensional information into the often foggy appearance of texture-based volume renderings. Although the method does not perform lighting calculations, the resulting image has a shaded appearance, which is a further visual cue to spatial understanding of the data and lets the images appear more realistic.
As calculating the shadows is independent of the visualization process it can be applied to any form of volume visualization, though volume rendering based on two- or three-dimensional texture mapping hardware makes the most sense. Compared to unshadowed texture-based volume rendering, performance decreases by less than 50%, which is still sufficient to guarantee interactive manipulation of the volume data. In the special case where only the camera is moving with the light position fixed to the scene there is no performance decrease at all, because recalculation has only to be done if the position of the light source with respect to the volume changes.
It is an object of the present invention to provide a comparatively simple method to accurately render shadowed semitransparent volumes with only a moderate increase in rendering time. The method is fast, easy to implement, and can be extended to multiple light sources.
According to the invention, this object is solved by a method for processing volumetric image data of a semitransparent object generating a shadow due to light of a light source impinging onto the object in a direction defined by a light vector, wherein each volumetric image data represents a value of transparency and a value of brightness and wherein the method comprises:
(a) dividing the volumetric image data into several image slice data representing individual layers of the object arranged sequentially behind each other,
(b) on the basis of the light vector and the values of transparency of the image slice data of the object layer being closest to the light source, generating a shadow value for each image slice data of the object layer being closest to the light source and storing these shadow values,
(c) overlaying the stored shadow values generated in step (b) and the brightness values of the image slice data of the object layer second most close to the light source according to the light vector and the distance between these two adjacent object layers,
(d) on the basis of the light vector and the values of transparency of the image slice data of the object layer second most close to the light source, generating shadow values for each image slice data of the second most close object layer and adding these values to the stored shadow values,
(e) overlaying the stored shadow values generated in step (d) and the brightness values of the image slice data of the object layer third most close to the light source according to the light vector and the distance between these two adjacent object layers,
(f) performing steps (d) and (e) for each object layer which is next in the direction away from the light source, and
(g) volume rendering the brightness values of the image slice data of the object layer closest to the light source and the overlaid shadow and brightness values of the image slice data of all the other object layers to display the semitransparent object including the shadow generated by the semitransparent object.
Preferably, overlaying comprises a step of blending the brightness and shadow values according to a blending function.
According to the invention, the shadows cast by the individual volumetric image data (voxels) are calculated successively from layer to layer of the object. In order to render a volume, the volumetric image data are divided into several groups of image slice data wherein the groups represent individual layers of the object arranged sequentially behind each other. According to the invention, there is calculated the accumulated shadow generated by the individual image slice data of the layers. Each volumetric image data represents both a value of transparency and brightness. Preferably to each volumetric image data a brightness value is associated wherein the transparency value can be regarded as the inverse of the brightness value.
After having calculated the shadow values of each image slice data of a respective layer, these shadow values are combined with the brightness values of the image slice data of the object layer being next in the direction of the light vector. These overlaid shadow and brightness values for each object layer are stored. After all the image slice data of all the layers are processed in that the shadows cast by them onto adjacent layers are calculated, the stored overlaid shadow and brightness values are used for rendering the shadowed volume/object. In this rendering process for showing the image slice data of the layer closest to the light source, the brightness values of its image slice data are used. Namely, there is no shadow which cast onto this layer.
When overlaying the generated shadow values of the image slice data of a layer and the brightness values of the image slice data of the layer next in the direction of the light vector, the orientation of the light vector and, in particular, the angle of the direction of the light vector and the viewing direction, as well as the values of transparency of the image slice data, are taken into consideration. The location at which a shadow resulting from the transparency value of an image slice data hits the layer next in the direction of the light vector is calculated on the basis of the orientation of the light vector and the distance between two adjacent object layers. This distance is a distance between the centers of two voxels of two adjacent layers.
The advantage of the method according to the invention is that it is based on the well-known texture-based volume rendering applied which according to the invention is to a semitransparent object/volume including shadows. In that the individual parts of the overall shadow generated by the object are calculated on a layer-to-layer-basis, the results again are layer-oriented image slice data, which can be rendered using texture mapping.